Systems and methods for automated diagnosis and decision support for breast imaging

ABSTRACT

CAD (computer-aided diagnosis) systems and applications for breast imaging are provided, which implement methods to automatically extract and analyze features from a collection of patient information (including image data and/or non-image data) of a subject patient, to provide decision support for various aspects of physician workflow including, for example, automated diagnosis of breast cancer other automated decision support functions that enable decision support for, e.g., screening and staging for breast cancer. The CAD systems implement machine-learning techniques that use a set of training data obtained (learned) from a database of labeled patient cases in one or more relevant clinical domains and/or expert interpretations of such data to enable the CAD systems to “learn” to analyze patient data and make proper diagnostic assessments and decisions for assisting physician workflow.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. Provisional Application Ser.No. 60/482,293, filed on Jun. 25, 2003, and U.S. Provisional ApplicationSer. No. 60/541,360, filed on Feb. 3, 2004, both of which are fullyincorporated herein by reference.

TECHNICAL FIELD OF THE INVENTION

The present invention relates generally to systems and methods forproviding automated diagnosis and decision support for medical imagingand, in particular, to CAD (computer-aided diagnosis) systems andapplications for breast imaging, which use machine-learning techniquesthat enable such systems and application to “learn” to analyzeparameters extracted from image data and/or non-image patient data of asubject patient for purposes of providing automated decision supportfunctions to assist a physician in various aspects of physician workflowincluding, but not limited to, diagnosing medical conditions (breasttumors) and determining efficacious healthcare or diagnostic ortherapeutic paths for the subject patient.

BACKGROUND

Today, in most countries, women over a certain age (usually 40) arescreened for breast cancer using X-ray mammography. If the results ofthe X-ray mammography present suspicious or potentially cancerous breasttissue, the patient is sent for a diagnostic workup. Alternatively, thepatient can be sent for a diagnostic workup through other paths, such asthe result of a physical examination in which the examining physicianfeels or otherwise identifies some abnormal feature (e.g., lump) in apatient's breast, or in circumstance in which the patient has anextremely high risk of cancer as determined through the patient'sclinical, history, or other means.

In a diagnostic workup, the patient's breasts will be imaged with one ofseveral imaging modalities, including X-ray mammography (digital oranalog), MRI, or ultrasound, for the purposes of screening or evaluatingfor anatomical abnormalities in breast tissue includingmicrocalcifications or masses in breast tissue, and various otherlesions or abnormalities that are potentially cancerous. Newertechniques are also being developed for diagnostic purposes, includingX-ray tomosynthesis, optical imaging, strain imaging, nuclear imaging,etc, which can be used to obtain diagnostic images of the patient'sbreast for evaluation by the physician determine whether a particularlesion in breast tissue is benign or malignant.

After reviewing a diagnostic image, if the physician believes that alesion may be malignant, a biopsy will be performed to remove a piece ofthe lesion tissue for analysis. This process is assumed to be a “goldstandard” for characterization of benign or malignant tissue. However,it is preferable to minimize the number of biopsies that are performedfor various reasons. For instance, a biopsy procedure causes pain andscarring for the patient, and the long period of time between the timeof the biopsy procedure and the time the results are provided to thepatient (usually at least a few days), the patient may be becomeseverely stressed in anticipation of potentially obtaining negativeresults. On the other hand, biopsy procedures enable physicians toaccurately diagnose a large percentage of patients with breast cancer.Thus, there is some trade-off or balance between sensitivity andspecificity that is typically maintained.

In the field of medical imaging, although various imaging modalities andsystems can be used for generating diagnostic images of anatomicalstructures for purposes of screening and evaluating medical conditions,with respect to breast cancer detection, each diagnostic imagingmodality has its own advantages and disadvantages, and the optimalchoice of imaging modality may not be the same for every patient.Ideally, the imaging modality for a given patient is selected tomaximize sensitivity and specificity for the patient. For each patient,there may be one or more “optimal” imaging modalities for such purpose.Unfortunately, due to cost, it is not possible to image every patientusing multiple imaging modalities, and then choose which modality wouldprovide the optimal balance between sensitivity and specificity.

The choice of diagnostic imaging modality is usually made by thereferring physician based on a number of factors, including, forexample, (i) availability and cost, (ii) comfort level and experience ofthe referring physician, or (ii) a physician's “gut feeling” as to whichimaging modality would be optimal to obtain information for the patient.While the first factor is unavoidable, the second and third factors canlead to a sub-optimal choice of imaging modality for the individualpatient.

SUMMARY OF THE INVENTION

Exemplary embodiments of the invention generally include systems andmethods for providing automated diagnosis and decision support forbreast imaging. More specifically, exemplary embodiments of theinvention include CAD (computer-aided diagnosis) systems andapplications for breast imaging, which implement automated methods forextracting and analyzing relevant features/parameters from a collectionof patient information (including image data and/or non-image data) of asubject patient to provide automated assistance to a physician forvarious aspects of physician workflow in breast care. For example, a CADsystem can provide automated diagnosis of breast cancer and otherrelated conditions, assessments with regard to the risk of a subjectpatient having breast cancer and/or developing breast cancer in thefuture, and other automated decision support functions to assist aphysician in determining efficacious healthcare or diagnostic ortherapeutic paths for a subject patient based on a current state of thepatient.

In other exemplary embodiments of the invention, CAD systems and methodsfor breast imaging implement machine-learning techniques which usetraining data that is obtained (learned) from a database of previouslydiagnosed (labeled) patient cases in one or more relevant clinicaldomains and/or expert interpretations of such data to enable the CADsystems to “learn” to properly and accurately analyze patient data andmake proper diagnostic and/or therapeutic assessments and decisions forassisting physician workflow.

These and other exemplary embodiments, features and advantages of thepresent invention will be described or become apparent from thefollowing detailed description of exemplary embodiments, which is to beread in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system for providing automatic diagnosticand decision support for breast imaging according to an exemplaryembodiment of the invention.

FIG. 2 is a block diagram of a system for providing automatic diagnosticand decision support for breast imaging according to another exemplaryembodiment of the invention.

FIG. 3 is a block diagram of a system for providing automatic diagnosticand decision support for breast imaging according to another exemplaryembodiment of the invention.

FIG. 4 is a block diagram of a system for providing automatic diagnosticand decision support for breast imaging according to another exemplaryembodiment of the invention.

FIG. 5 is an exemplary diagram illustrating a classification methodaccording to an exemplary embodiment of the invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

In general, exemplary embodiments of the invention as described belowinclude systems and methods for providing automated diagnosis anddecision support for breast imaging. More specifically, exemplaryembodiments of the invention as described below with reference to FIGS.1˜4, for example, include CAD (computer-aided diagnosis) systems andapplications for breast imaging, which implement automated methods forextracting and analyzing relevant features/parameters from a collectionof patient information (including image data and/or non-image data) of asubject patient to provide automated assistance to a physician forvarious aspects of physician workflow including, for example, automatedassistance to a physician for various aspects of physician workflowwhere decisions must be made respecting healthcare or diagnosis pathsand/or therapeutic paths for the patient. Various methods have beendeveloped which attempt to provide decision support for physicians usingonly information from images. However, these techniques ignore the factthat there is a significant amount of information contained in thepatient record in the form of non-image data. Advantageously, asdescribed in detail below, CAD systems and methods according toexemplary embodiments of the invention provide automated decisionsupport methods that combine both imaging and non-imaging data. Here,non-imaging data is taken to include all information found in apatient's record other than images, which can include but not be limitedto, demographic data, history and physical information, physician notes,lab results, results from blood tests, results from proteomic analysis,and results from genetic assays. For example, in the specific case ofbreast imaging, two women with identical images with suspicions findingsmay be treated differently if, for example, one patient is a young womanwith no history or risk factors for cancer, while the other patient isan elderly woman with genetic disposition for breast cancer (such as thepresence of the BRCA gene) and a known family history of breast cancer.Combining the clinical and imaging information provides the mostvaluable assistance for the physician.

For instance, given a set of information that is collected for a givenpatient, CAD systems according to exemplary embodiments of the inventioncan extract and analyze relevant features from such patient informationto automatically assess the current state of the patient (e.g.probability and confidence of diagnosis of a disease or a likelihood ofhaving a particular disease given history, age, etc.), automaticallydetermine which additional test(s) or features(s), if any, would beuseful to increase the confidence in a diagnosis, and otherwise providedecision support to a physician in other aspects of physician workflow.

Exemplary CAD systems and applications according to the inventionimplement machine-learning techniques that use training data obtained(learned) from a database of labeled patient cases in one or morerelevant clinical domains and/or expert interpretations of such data toenable the CAD systems to “learn” to properly and accurately analyzepatient data and make proper diagnostic assessments and decisions forassisting physician workflow. For example, with respect to breastimaging a diagnosis of breast cancer, exemplary CAD systems describedbelow can “learn” to provide proper assessments in the areas ofscreening, diagnosis and/or staging of breast cancer. For illustrativepurposes, exemplary embodiments of the invention will be described withspecific reference to breast imaging and physician workflow for breastcare. It is to be understood, however, that the present invention is notlimited to any particular medical fields. Rather, the invention is moregenerally applicable to any medical field of practice in which physicianworkflow requires the physician to determine or assess the current stateof a patient and determine workflow paths would result in a moreaccurate assessment of the current state of the patient for purposes ofproviding the appropriate care. Those of ordinary skill in the art willreadily appreciate that CAD systems according to exemplary embodimentsof the invention provide a powerful tool to assist physician workflow.

It is to be understood that the systems and methods described herein inaccordance with the present invention may be implemented in variousforms of hardware, software, firmware, special purpose processors, or acombination thereof. In one exemplary embodiment of the invention, thesystems and methods described herein are implemented in software as anapplication comprising program instructions that are tangibly embodiedon one or more program storage devices (e.g., magnetic floppy disk, RAM,CD Rom, DVD, ROM and flash memory), and executable by any device ormachine comprising suitable architecture, wherein the application may bea distributed network application with n-tier client-server architecturefor a distributed network application, etc.

It is to be further understood that because the constituent systemmodules and method steps depicted in the accompanying Figures can beimplemented in software, the actual connections between the systemcomponents (or the flow of the process steps) may differ depending uponthe manner in which the application is programmed. Given the teachingsherein, one of ordinary skill in the related art will be able tocontemplate these and similar implementations or configurations of thepresent invention.

FIG. 1 is a high-level block diagram of a system for providing automateddiagnostic support and physician workflow assistance for breast imaging,according to an exemplary embodiment of the invention. Morespecifically, FIG. 1 illustrates a CAD (computer-aided diagnosis) system(10) that implements methods for analyzing various types of patientinformation (1) and (2) of a subject patient to provide diagnosticassessments and recommendations and other decision support to assist aphysician in various aspects of physician workflow with respect to thesubject patient. The CAD system (10) uses machine learning methods thatenable the CAD system (10) to continually learn to analyze the patientinformation (1, 2) and continually provide more accurate diagnosticassessments and/or decisions to assist physician workflow.

The input to the CAD system (10) comprises various sources of patientinformation including image data (1) in one or more imaging modalities(e.g., ultrasound image data, X-ray mammography image data, MRI etc.)and non-image data (2) from various structured and/or unstructured datasources, including clinical data which is collected over the course of apatient's treatment and other information such as patient history,family history, demographic information, financial information, and anyother relevant patient information. The CAD system (10) implementsmethods for automatically extracting information (features) from theimage data (1) and non-image data (2) and combining the extractedinformation in a manner that is suitable for analysis by the CAD system(10). Depending on the diagnostic and decision support function(s)supported by the CAD system (10), the CAD system (10) can generate oneor more outputs (11), (12), and/or (13). As explained below, in thefield of breast care, these outputs can provide physician workflowassistance in the areas of screening, diagnosing and/or staging forbreast cancer.

In another exemplary embodiment of the invention, the CAD system (10)can extract and analyze information from image data (1) and (optionally)non-image data (2) to automatically generate and output a probability ofdiagnosis and (optionally) a measure of confidence of the diagnosis (11)or alternatively output a suggested therapy with a probability and(optional) measure of confidence as to the impact of the suggestedtherapy, e.g., the probability that the suggested therapy will have thedesired (beneficial) impact. Collectively, the output (11) can bereferred to herein as “Probability and Confidence of Suggestion”.

More specifically, by way of example, for purposes of diagnosing breastcancer, the CAD system (10) may comprise methods for automaticallydetecting and diagnosing (or otherwise characterizing) suspect breastlesions in breast tissue and outputting, for example, a probability ofmalignancy of such lesions, together with an optional measure ofconfidence in such diagnosis. In this example, the CAD system (10) couldextract and analyze relevant features from a screening X-ray mammogram(image data) and clinical history information (non-image data) of apatient and provide a current estimate and confidence of malignancy.

Alternatively, for patients with known breast cancer for example, theCAD system (10) could suggest an course of therapy, in which case, theprobability and confidence (11) would refer to the likelihood that thetherapy would have the desired (presumably beneficial) impact, whichcould range from curing the patient from breast cancer, to a purelypalliative treatment whose sole aim would be to improve the quality oflife of a patient with terminal breast cancer. More specifically, theCAD system (10) could in addition to suggesting a therapy, automaticallyprovide a probability and/or measure of confidence that the therapy willhave a determined outcome and possible provide a probability and/ormeasure of confidence that the therapy will not have a determineddetrimental impact such as side effects. The probability can bespecified as a distribution over possible outcomes both beneficial anddetrimental, or a set of distributions over possible outcomes bothbeneficial and detrimental at one or more time points in the future, ora time-varying distribution over possible outcomes at different times inthe future, etc.

In another exemplary embodiment of the invention, the CAD system (10)can automatically determine and specify one or more additional tests,measurements, or features which, if made/obtained, could increase theconfidence of diagnosis (i.e., sensitivity analysis). For example, theCAD system (10) can determine and output a “score” (12) for eachadditional test, measurement or feature, which provides some measure orindication as to the potential usefulness of the particular imagingmodality or feature(s) that would improve the confidence of anassessment or diagnosis determined by the CAD system (10). For example,assuming the CAD system (10) extracts and analyzes relevant featuresfrom a screening X-ray mammogram (image data) and clinical historyinformation (non-image data) of a patient and provides a currentestimate and confidence of malignancy of a detected lesion, the CADsystem (10) could further indicate which imaging modality or modalitieswould most likely provide the maximum amount of additional informationthat would be useful in determining whether the lesion is malignant orbenign, or determining the extent of cancer (“staging”), or would bemost useful in deciding on a course of therapy for a patient with knownbreast cancer—for instance, deciding between surgery, radiotherapy,chemotherapy, hormone therapy or some combination thereof (the so called“cocktail” therapy).

In another exemplary embodiment of the invention, the CAD system (10)can identify and output (via display or list) one or more exemplary casestudies that are similar to a current case (13). For example, as notedabove and explained in further detail below, the CAD system (10) maycomprise a database (or library) of previously labeled (diagnosed)cases, and based on features extracted from patient information input tothe CAD system (10) for the subject patient, the CAD system (10) cansearch and display the n-most relevant cases from the library fordiagnostic assistance. In other words, the CAD system (10) can provide aset of similar cases from the training set using the automaticallyextracted features.

It is to be appreciated that the CAD system (10) function of displayingsimilar cases in the context of physician workflow can providesignificant assistance to the physician. For instance, displayingsimilar cases can provide training for inexperienced users. Indeed,novice users can review other cases to determine or otherwise understandthe basis or reasons why the case interpreted in the way that it was.Moreover, display of similar cases can provide a means for experiencedusers to confirm the diagnostic results of the CAD system (10). Indeed,in addition to probability of diagnosis for a given condition, the CADsystem (10) could display similar cases to justify its assessment.Moreover, displaying similar cases enables assessment of prognosis andtreatment. More specifically, by studying similar cases to see how otherpatients responded to different treatment options, a physician can beginto assess the efficacy of these options for the current patient.

In view of the above, the CAD system (10) can be generally viewed as anautomated system that can assist physician workflow by providing anassessment of the current state of a patient (e.g. probability oflikelihood of a particular disease) and determining next best healthcare or diagnostic paths for the subject patient (e.g., identifyingadditional tests (or features) that can be obtained, which would likelyreduce any ambiguity of the assessment). As noted above, it is to beappreciated that the CAD system (10) implements one or moremachine-learning methods whereby the information is learned, and thedecisions driven, by data that is collected in a training set of the CADsystem (10). In particular, as noted above, the CAD system (10) couldinclude a library of exemplary diagnosed cases from which training datais obtained to teach the CAD system (10). In contrast to “expertsystems” which are developed and derived from a set of rules dictated byan expert and translated into code, the CAD system (10) learns toprovide accurate diagnostic decisions and provide decision support basedon training data that is learned from diagnosed cases or learned fromexpert knowledge.

It is to be appreciated that various machine learning methods may beimplemented by the CAD system (10). For example, the systems and methodsdescribed in U.S. patent application Ser. No. 10/702,984, filed on Nov.6, 2003, by Zhou et al, entitled “System and Method for Real-TimeFeature Sensitivity Analysis Based on Contextual Information,” which iscommonly assigned and incorporated herein by reference, can be used inthe CAD system (10) for determining which tests or features may be mostrelevant for reducing ambiguity of a diagnosis. Essentially, the Zhouapproach is to create a model of the process, and determine the relativeimportance of each feature in reducing ambiguity. Such method can beimplemented herein whereby each imaging modality, or diagnostic path,could be described as a set of one or more features. Then, the methodsdescribed by Zhou would be used to determine which feature(s) wouldlikely provide the greatest improvement in confidence in a diagnosis orassessment. Other machine learning techniques which learn from a largetraining set of cases can be implemented in the CAD system (10). Forexample, various machine learning techniques, such as decision trees,SVM, neural networks, or Bayesian networks, or ensemble methods whichcombine multiple such methods, for example, may be used. Alternately,model-based algorithms which would be defined or trained specifically todetect some kind of lesion, for instance, based on causal knowledge ofthe various factors that are related to a particular kind of lesion, forexample.

It is to be appreciated that the CAD system (10) can provide properdecision support even in the absence of various features or informationthat can be used for rendering such decisions. For example, if themedical records of a patient only contain a screening mammogram andbasic demographic information about the patient (for example, age andrace), but no clinical or family information, the CAD system (10) isable to provide a probability and confidence of diagnosis, along with abest estimation of what test or procedure should be performed next. Inthis case, the recommended procedure might even be to collect the familyinformation for the patient. Of course, the confidence of the systemwill improve with more information as it is provided. As an extreme,consider the situation where there no information at all for a givenpatient. In this instance, the CAD system (10) should be able to providea physician with some guidance as to an initial step to take withrespect to the patient. Various methods for learning and/or performinginference with missing/noisy data may be used in the decision supportsystem.

It is to be appreciated that the above methods can be extended toprovide automatic screening for medical conditions such as breastcancer. In the United States, current recommendations provide that allwomen over the age of 40 are to be screened yearly using X-raymammography. As in the case of breast cancer diagnosis, there have beenstudies in the literature to automatically assess the risk associatedwith mammograms. In this regard, it may be that MRI or ultrasound may bebetter screening tools for a particular subset of the population or forwomen with some particular finding in their diagnostic mammogram.Furthermore, for some women, the risk of developing breast cancer at thecurrent point in their life may be so small that it may not be worth therisk of being subjected to ionizing radiation, or the cost, to evenperform a screening.

Accordingly, the CAD system (10) can be configured to make adetermination, in view of a patient's clinical and family history, as tothe likelihood that the patient has (or can develop) breast cancer, andwhat screening test (if any) should be given to the patient to bestdetect suspicious lesions or risk of cancer for further diagnosis. Thislikelihood could also be inferred at any point in time during thepatient history, e.g., after the first screening exam, or after multiplescreens and MRI tests. Such determinations can be made using a trainingset as described above and using machine-learning techniques. Moreover,for screening, the CAD system (10) can generate and output decisions asdiscussed above, including likelihood of disease, exemplar cases from atraining set, and the screening test that would be optimal for the givenpatient. In this case, a determination as to the screening test may beof most interest. Indeed, for such determination, a screening mammogramwould not be available for the classification. Moreover, the comparisonwould not necessarily be made to correct a diagnosis of the patient, butrather to correct identification of either suspicious lesions in thebreast, or sufficient risk of breast cancer to warrant furtherdiagnostic tests.

In another exemplary embodiment of the invention, the CAD system (10)can provide assistance in breast imaging with regard to staging oftumors for therapy. In general, a staging process involves preciselylocating a lesion and determining if a lesion is single- or multi-focal.In according the an exemplary embodiment of the invention, the CADsystem (10) can learn to determine which test should be used to stage alesion, given information about the lesion obtained from screeningand/or diagnosis test. For example, in a training set, the results ofthe staging from different modalities could be compared to those resultsactually found during therapy or follow-up visits. Accordingly,machine-learning methods as described above can be used to enable theCAD system (10) to “learn” a proper approach to staging for a givenpatient. Exemplar cases from the training set can also potentially showwhat the results of the staging, and perhaps even the outcomes aftertherapy for patients with “similar” cases.

The exemplary CAD systems and methods discussed above with reference toFIG. 1 provide a general framework for developing CAD systems that cansupport one or more imaging modalities and provide one or morefunctionalities for providing assistance in various aspects of physicianworkflow. Exemplary embodiments of CAD systems and methods according tothe invention, which are based on the framework of FIG. 1, will bediscussed with reference to FIGS. 2, 3 and 4, for example, for providingassistance to physician workflow in breast imaging. The exemplaryembodiments of FIGS. 2 and 3 depict CAD systems and methods for breastimaging for one or more ultrasound imaging modalities. FIG. 4 is anexemplary embodiment of a CAD system which incorporates the systems ofFIGS. 2 and 3 and provides further functionality for enabling amulti-modal CAD system that can be used for various for breast imagingin multiple imaging modalities.

Referring now to FIG. 2, a block diagram illustrates a system forproviding automatic diagnostic and decision support for breast imagingaccording to another exemplary embodiment of the invention. Inparticular, the CAD system (20) of FIG. 2 illustrates one or moreexemplary frameworks for implementing the CAD system (10) of FIG. 1 tosupport ultrasound (B-mode analysis) breast imaging. In general, the CADsystem (20) comprises a data processing system (21) which comprises afeature extraction module (22), a feature combination module (23), aclassification module (24), a diagnostic/workflow assistance module (25)and an automated detection module (29).

The automated detection module (29) implements methods for processingultrasound image data (3) of breast tissue to detect and segmentpotential lesions in the imaged breast tissue. More specifically, theautomated detection module (29) implements one or more conventionalmethods for processing ultrasound image data (3) to automatically detectlesions and other abnormal anatomical structures such as microcalcifications or masses in breast tissue, etc. The automated detectionmodule (29) automatically detects and mark regions of features ofinterest in the image data, which are identified as being potentiallesions, abnormalities, disease states, etc.

The feature extraction module (22) implements various methods (22-1,22-2, 22-3, 22-4) for extracting relevant parameters from ultrasoundimage data (3) and other sources of non-image patient data (4) such asclinical, family, history data, etc, such as described in further detailbelow, which can be used for providing automated diagnosis and decisionsupport functions. The feature combination module (23) combines theextracted features in a manner that is suitable for input to theclassification module (24) for analysis.

The classification module (24) comprises a classification method (24-1)(or classification engine) that analyzes the combined extractedparameters using one or more classification models, which aretrained/dynamically adapted via model builder (24-2), to generateinformation that is used to provide diagnostic and decision support. Thediagnostic/workflow assistance module (25) includes one or more methodsfor implementing functions such as described above with reference toFIG. 1 (e.g., providing a diagnosis, providing a set of cases similar toa current case, providing a score showing the likely benefit ofadditional tests or features that would improving the confidence ofdiagnosis, etc.).

The CAD system (20) further comprises a user interface (26) (e.g.,graphical user interface displayed on computer monitor with keyboard andmouse input devices) which enables a user to select one or morefunctions supported by the diagnostic/workflow assistance module (25)and which enables the system to render and present processing results tothe user. The processing results can be rendered and presented to a userin one or more of various ways according to exemplary embodiments of theinvention as described below.

The CAD system (20) further comprises a repository (27) that maintains aclinical domain knowledge base of information that is derived fromvarious sources. For instance, the clinical domain knowledge (27) mayinclude knowledge that is learned or automatically extracted from alarge database of analyzed/labeled cases (28) related to the clinicaldomain(s) supported by the CAD system (20). The clinical domainknowledge (27) may include expert clinical knowledge that is inputdirectly by an expert from analyzing previous claims, or informationrelated to rules/regulations/guidelines associated with medical bodiesor insurance companies, with regard to the supported clinical domain(s).As explained in detail below, the clinical domain knowledge inrepository (27) can be used by the various methods (22, 23, 24, and 25)of the data processing system (21).

The feature extraction module (22) includes various methods to extractimage parameters associated with the “detected” regions of theultrasound image data, which can be used diagnosing potential canceroustissue. Such features include parameters associated with speculation(22-1), acoustic shadowing (22-2), height/depth ratio (22-3) and/orother possible image parameters that can be used to automaticallyclassify lesions or abnormalities in breast tissue.

In other exemplary embodiments of the invention, the data processingsystem (21) extracts and analyzes relevant parameters from non-imagepatient data records (4) of a subject patient, which can used inconjunction with the extracted image parameters (22-1, 22-3, 22-3) toprovide automated diagnosis. The patient data (4) can include patientinformation from a plurality of structured and unstructured datasources, which is collected over the course of a patient's treatment. Ingeneral, the structured data sources include, for example, financial(billing), laboratory, and pharmacy databases, wherein patientinformation in typically maintained in database tables. The unstructureddata sources include for example, waveform data, free-text baseddocuments of laboratory test results, doctor progress notes, detailsabout medical procedures, prescription drug information, radiologicalreports, and other specialist reports.

The non-image patient data (4) can include a significant amount ofuseful data indicative of a person having breast cancer or a historythat indicates that the person has a high potential for developingbreast cancer. By way of example, such clinic information may be foundin history and physical notes, wherein a physician notes that a personhas been previously diagnosed with breast cancer. Other indications,such as family history of breast cancer, history of smoking, age,gender, etc., can also be used to assess the risk of developing orhaving breast cancer. Accordingly, the feature extraction module (22)includes one or more data extraction methods (22-4) for extractingrelevant patient data from the non-image patient data (4), which may berelevant for assessing or diagnosing a medical condition.

It is to be appreciated than any suitable data analysis/data miningmethods may be implemented by the extraction module(s) (22-4) forextracting relevant parameters from the patient data (4). In oneexemplary embodiment of the invention, patient data extraction methods(22-4) and feature combination method (23) may be implemented using thedata mining methods and feature combination methods as described incommonly assigned and copending U.S. patent application Ser. No.10/287,055, filed on Nov. 4, 2002, entitled “Patient Data Mining”, whichclaims priority to U.S. Provisional Application Ser. No. 60/335,542,filed on Nov. 2, 2001, which are both fully incorporated herein byreference. Briefly, U.S. Ser. No. 10/287,055 describes data miningmethods for extracting relevant information from clinical data recordsusing domain-specific knowledge contained in a knowledge base (e.g., inrepository (27)), which are represented as probabilistic assertionsabout the patient at a particular time (referred to as elements) andcombining all elements that refer to the same variable (domain-specificcriteria) at a given time period to form a single unified probabilisticassertion regarding that variable.

In the exemplary embodiment of FIG. 2, as noted above, the dataprocessing system (22) uses clinical domain knowledge data maintained inthe repository (27) to perform the various methods of feature extraction(22), feature combination (23) and model building (24-2). Thedomain-specific knowledge base (27) may include disease-specific domainknowledge. For example, the disease-specific domain knowledge mayinclude various factors that influence risk of a disease, diseaseprogression information, complications information, outcomes andvariables related to a disease, measurements related to a disease, andpolicies and guidelines established by medical bodies such as theAmerican College of Radiology (ACR). The domain-specific knowledge base(27) may also include institution-specific domain knowledge. Forexample, this may include information about the data available at aparticular hospital, document structures at a hospital, policies of ahospital, guidelines of a hospital, and any variations of a hospital.

The clinical domain knowledge base (27) may be derived from varioussources. For instance, the clinical domain knowledge base (27) mayinclude knowledge that is learned from a large database ofanalyzed/labeled cases (28). In addition, the clinical domain knowledgebase (27) may include knowledge that is input by an expert fromanalyzing previous claims, or from rules and regulations published by aninsurance company, for example. The data in the domain knowledge base(27) can be encoded as an input or as programs that produce informationthat can be understood by the system. As noted above, the domain expertdata may be obtained by manual input from a domain expert using anappropriate user interface or the domain expert data may beautomatically or programmatically input.

The extraction modules (22-4) can use relevant data in the domainknowledge base (27) to extract relevant parameters and produceprobabilistic assertions (elements) about the patient that are relevantto an instant in time or time period. The domain knowledge required forextraction is generally specific to each source. For example, extractionfrom a text source may be carried out by phrase spotting, wherein a listof rules are provided that specify the phrases of interest and theinferences that can be drawn therefrom. For example, if there is astatement in a doctor's note with the words—“There is evidence oflesions in the left breast”—then, in order to infer from this sentencethat the patient has or may have breast cancer, a rule can be specifiedthat directs the system to look for the phrase “lesion,” and, if it isfound, to assert that the patient may have breast cancer with a somedegree of confidence. Extraction from a database source may be carriedout by querying a table in the source, in which case, the domainknowledge needs to encode what information is present in which fields inthe database. On the other hand, the extraction process may involvecomputing a complicated function of the information contained in thedatabase, in which case, the domain knowledge may be provided in theform of a program that performs this computation whose output may be fedto the rest of the system.

The methods implemented by the feature combination module (23) can bethose described in the above-incorporated patent application. Forexample, a feature combination method can be a process of producing aunified view of each variable at a given point in time from potentiallyconflicting assertions from the same/different sources. In variousembodiments of the present invention, this is performed using domainknowledge regarding the statistics of the variables represented by theelements.

The model builder (24-2) builds classification models implemented by theclassification method (24-1), which are trained (and possiblydynamically optimized) to analyze various extracted features and providediagnostic assistance and assessment on various levels, depending on theimplementation. It is to be appreciated that the classification modelsmay be “black boxes” that are unable to explain their prediction to auser (which is the case if classifiers are built using neural networks,example). The classification models may be “white boxes” that are in ahuman readable form (which is the case if classifiers are built usingdecision trees, for example). In other embodiments, the classificationmodels may be “gray boxes” that can partially explain how solutions arederived (e.g., a combination of “white box” and “black box” typeclassifiers). The type of classification models that are implementedwill depend on the domain knowledge data and model building process(24-2). The type of model building process will vary depending on theclassification scheme implemented, which may include decision trees,support vector machines, Bayesian networks, probabilistic reasoning,etc., and other classification methods that are known to those ofordinary skill in the art.

The model builder/update process (24-2) uses data in the clinical domainknowledge base (27) to train classification models, and possiblydynamically update previously trained classification models that areimplemented by the classification process (24-1). In one exemplaryembodiment of the invention, the model builder/update process (24-2) isimplemented “off-line” for building/training a classification model thatlearns to provide proper diagnostic assessments and decisions forworkflow assistance. In another exemplary embodiment of the invention,the model builder/update process (24-2) employs “continuous” learningmethods that can use the domain knowledge data in repository (27) whichis updated with additional learned data derived from newly analyzedpatient data or otherwise optimize the classification model(s)associated with the relevant condition. Advantageously, a continuouslearning functionality adds to the robustness of the CAD system (20) byenabling the classification process (24-1) to continually improve overtime without costly human intervention.

The diagnostic/workflow assistance module (26) can provide one or morediagnostic and decision support functions as described above withreference to FIG. 1. For instance, the diagnostic/workflow assistancemodule (26) can command the classification module (24) to classify oneor more breast lesions detected in ultrasound image data (4) asmalignant or benign and provide a probability of such diagnosis and(optionally) a measure of confidence in the diagnosis, based on a set offeatures extracted from ultrasound image data (3) and/or non-imagepatient data records (4). The classification engine (25-1) could performsuch classification using one or more classification models that aretrained to analyze the combined features output from module (23). Inanother exemplary embodiment, the diagnostic/workflow assistance module(25-1) can command the classification module (24) to determine whatadditional image parameter or features (e.g., from B-mode ultrasoundimage data, other image mode, and/or non-image data) can be obtained andfurther analyzed to increase the confidence in the diagnosis. Moreover,the diagnostic/workflow assistance module (25-1) can command theclassification module (23) to obtain and display (via user interface)one or more similar patient cases in repository (27) based on thecurrent set of extracted features.

Referring now to FIG. 3, a block diagram illustrates a system forproviding automated diagnostic and decision support for breast imagingaccording to another exemplary embodiment of the invention. Morespecifically, FIG. 3 illustrates a CAD system (30) that supportsadditional ultrasound imaging methods (in addition to B-mode analysis)for providing automated diagnosis of breast lesions in breast tissue,for example, and other decision support function to assist physicianworkflow. In one exemplary embodiment, the CAD system (30) of FIG. 3incorporates an automated B-mode analysis of the CAD system (20)discussed above with reference to FIG. 2. The CAD system (30) of FIG. 3illustrates one or more exemplary frameworks for the CAD system (10) ofFIG. 1 to support one or more ultrasound imaging methods including, forexample, B-mode, contrast imaging, and/or strain imaging, etc.

More specifically, referring to FIG. 3, the CAD system (30) comprises adata processing system (31) which implements methods for automaticclassification (diagnosis) of breast cancer based on various parametersare extracted from one or more types of ultrasound image data (5) and/ornon-image patient data (6), as well as other methods to assist aphysician to decide an a care or diagnosis path for a particularpatient. In general, the data processing system (31) comprises a featureextraction module (32), a feature combination module (33), aclassification module (34) and a diagnostic/workflow assistance module(35). Moreover, the CAD system (30) comprises a user interface (36)which enables user interaction with the CAD system (30) to select one ormore functions supported by the diagnostic/workflow assistance module(35) (e.g., providing automated diagnosis and confidence of diagnosisfor breast cancer, determine what additional ultrasound imagingmodalities or features (e.g., from B-mode ultrasound image data, otherimage mode, and/or non-image data) can be obtained and further analyzedto increase the confidence in diagnosis, obtain and display one or moresimilar patient cases in a repository (38) based on the current set ofextracted features.)

The feature extraction module (32) implements various methods(32-1˜32-5) for extracting relevant parameters from one or more ofvarious modes of ultrasound image data (5) and non-image patient data(6), which can be analyzed to provided automated diagnosis and othertypes of decision support as discussed herein. For instance, the featureextraction module (32) includes an automated B-mode analysis module(32-1) which implements, for example, the automated detection (29),speculation (23-1), acoustic shadowing (23-2), and H/D ratio (23-3)methods as described above in the system (20) of FIG. 2. In addition,the feature extraction module (32) includes methods for extractingrelevant parameters from ultrasound measurements including strain andelastography (32-2), motion of fluid using techniques such as acousticstreaming (32-3), 3D ultrasound imaging (32-4) and motion of blood usingtechniques such as contrast perfusion (32-5).

The various feature extraction modules can be implemented using methodsthat are well known to those of ordinary skill in the art. For example,for ultrasound strain/elastography imaging, the systems and methoddescribed in the following patents: Hall et al, “Ultrasonic elasticityimaging”, U.S. Pat. No. 6,508,768, issued Jan. 21, 2003; Nightingale etal, “Method and apparatus for the identification and characterization ofregions of altered stiffness”, U.S. Pat. No. 6,371,912, issued Apr. 16,2002; and Von Behren et al, “System and method for strain imagedisplay”, U.S. Pat. No. 6,558,424, issued May 6, 2003, which are allincorporated herein by reference, can be implemented for extractingrelevant parameters from ultrasound measurements including strain andelastography. Moreover, the systems and methods for acoustic streamingas described in Trahey et al, “Method and apparatus for distinguishingbetween solid masses and fluid-filled cysts”, U.S. Pat. No. 5,487,387,issued Jan. 30, 1996, which is incorporated herein by reference, can beused for extracting features related to motion of fluid. In addition,the systems and methods for contrast perfusion as described in Philipset al, “Dual process ultrasound contrast agent imaging”, U.S. Pat. No.6,632,177, issued Oct. 14, 2003, which is incorporated herein byreference, may be used for extracting features related to motion ofblood. It is to be understood that other known techniques may beimplemented.

The feature combination module (33) combines a set of extracted featuresin a manner that is suitable for input and analysis by theclassification module (34). The classification module (34) comprisesclassification methods (34-1) to analyze the combined extractedparameters using one or more classification models, which aretrained/dynamically adapted via model builder (34-2), to provideautomatic diagnosis of breast cancer and other decisions supportfunctions. The CAD system (30) further comprises a repository (37) thatmaintains a clinical domain knowledge base of information which providestraining data used by the model builder (34-2) to build/trainclassification models used by the classification methods (34-1). A largedatabase of analyzed/labeled cases (38) related to the clinical domainor domains supported by the CAD system (30) can be used to obtaintraining data in repository (37). The clinical domain knowledge inrepository (37) can be used by the various methods (32, 33, 34 and 35)of the data processing system (31).

In general, the various components of the CAD system (30) of FIG. 3 areessentially similar to those of the CAD system (20) of FIG. 2 asdiscussed above, except that the CAD system (30) of FIG. 3 provides amore diverse framework that supports various ultrasound imaging methodsin addition to B-mode ultrasound to enable a more complete CAD systemfor ultrasound breast imaging. It is to be appreciated that the variousmodules (32, 33, 34 and 35) in FIG. 3 can implement the same or similarmethods as those corresponding modules (22, 23, 24 and 25) of the CADsystem (20) of FIG. 2 as described above. However, the various methods,such as the classification and model building methods in classificationmodules (24) and (34) will vary depending on the types of decisionsupport functions, feature extraction methods and/or image modalitiessupported by the respective CAD systems (20) and (30). Moreover, theclinical domain knowledge base (37) is similar to the knowledge base(27) of FIG. 2, except that the training data in knowledge bases (27)and (37) will vary depending on the types of decision support functions,feature extraction methods and/or image modalities supported by therespective CAD systems (20) and (30).

Referring now to FIG. 4, a block diagram illustrates a system forproviding automated diagnostic and decision support for breast imagingaccording to another exemplary embodiment of the invention. Morespecifically, in one exemplary embodiment of the invention, FIG. 4illustrates a CAD system (40) that is an extension of the exemplary CADsystems (20) and (30), wherein the CAD system (40) incorporates thefunctions and methods of CAD systems (20) and (30) for ultrasound breastimaging, and further incorporated methods and functions for enabling amulti-modal CAD for breast imaging in multiple imaging modalities.

Referring to FIG. 4, the CAD system (40) comprises a data processingsystem (41) which implements methods for providing automated diagnosisof breast lesions in breast tissue and providing decision support fordiagnostic and/or care paths to assist physician workflow, by extractingand analyzing parameters from various sources of patient information(7), including, for example, one or more different types of image data(e.g., MRI image data (7 a), ultrasound image data (7 b), X-Raymammography image data (7 c) and non-image data such as genetics and/orproteomics data (7 d) and clinical, history and/or physical data (7 e)of the subject patient.

In general, the data processing system (41) comprises a featureextraction module (42), a feature combination module (43), aclassification module (44) and a diagnostic/workflow assistance module(45). Moreover, the CAD system (40) comprises a user interface (46)which enables user interaction with the CAD system (40) to select one ormore functions supported by the diagnostic/workflow assistance module(45) (e.g., providing automated diagnosis and confidence of diagnosisfor breast cancer, determining what additional imaging modalities orfeatures could be obtained and further analyzed to increase theconfidence in diagnosis, obtaining and displaying one or more similarpatient cases in a repository based on a current set of extractedfeatures, etc.)

The feature extraction module (42) implements “n” feature extractionmethods for extracting image parameters (42-1˜42-2) from the supportedimaging modalities, and other feature or text extraction methods (42-3,42-4) for extracting parameters from non-image data sources. Forinstance, the feature extraction module (42) can include methods forextracting and analyzing image parameter from various types ofultrasound data (as discussed above with reference to FIGS. 2 and 3) andother imaging modalities. The feature combination module (43) combines aset of extracted features in a manner that is suitable for input andanalysis by the classification module (44). The classification module(44) comprises classification methods (44-1) to analyze the combinedextracted parameters using one or more classification models, which aretrained/dynamically adapted via model builder (44-2), to provide thevarious decision support functions. The CAD system (40) furthercomprises a repository (47) that maintains a clinical domain knowledgebase of information which provides training data used by the modelbuilder (44-2) to build/train classification models used by theclassification methods (44-1). A large database of analyzed/labeledcases (48) related to the clinical domain or domains supported by theCAD system (40) can be used to obtain training data that is stored inthe repository (47). The clinical domain knowledge in repository (47)can be used by the various methods (42, 43, 44 and 45) of the dataprocessing system (41).

It is to be appreciated that the various modules (42, 43, 44 and 45) inFIG. 4 can implement the same or similar methods as those correspondingmodules (22, 23, 24 and 25) of the CAD system (20) of FIG. 2 and/orcorresponding modules (32, 33, 34 and 35) of the CAD system (30) of FIG.3, as described above. However, the various methods, such as theclassification and model building methods of the classification module(44) will vary depending on the types of decision support functions,feature extraction methods and/or image modalities supported by the CADsystem (40). Moreover, the clinical domain knowledge base (47) issimilar to the knowledge bases (27) and (37) of FIGS. 2 and 3, exceptthat the training data in knowledge bases (47) will vary depending onthe types of decision support functions, feature extraction methodsand/or image modalities supported by the CAD system (40).

Various machine learning methods according to exemplary embodiments ofthe invention for assessing the likely value of additional tests fordiagnosis of breast cancer will now be described with reference to theexemplary node diagram of FIG. 5. For these exemplary embodiments, it isassumed that a training set consists of m cases and each case consistsof n features extracted from previously performed tests. Each caseC_(i), (i=1, . . . , m) can be represented as a vector of features (ƒ₁,ƒ₂, . . . , ƒ_(n)).

It is further assumed that for each case C_(i), the real diagnosis(d_(i)) given by a biopsy result is:

$d_{i} = \left\{ \begin{matrix}1 & {{If}\mspace{14mu} a\mspace{14mu}{lesion}\mspace{14mu}{is}\mspace{14mu}{malignant}} \\0 & {Otherwise}\end{matrix} \right.$and that there are k variables corresponding to the different tests thatwere performed on the patients (T_(i1), T_(i2), T_(i3), . . . T_(ik)),wherein each one of the k variables can take values in the set {0,1},and wherein k=1 if the corresponding test predicted correctly withrespect to the real diagnosis d_(i), or where k=0 otherwise.

Further assuming that such previous information is extracted from thetraining data, the exemplary machine learning based methods describedhereafter can be used to predict which test will provide an accuratediagnosis based on a feature vector extracted from a patient medicalhistory.

In one exemplary embodiment, one method is as follows. First, a mappingM is determined from the feature space to {(P₁, P₂, P₃, P₄)/P_(i)ε{0,1}}such that for every C_(i), M(C_(i))=M(ƒ₁, ƒ₂, . . . , ƒ_(n))=(T_(i1),T_(i2), T_(i3), T_(i4)). This process can be achieved using artificialneural network techniques as illustrated in FIG. 5. For each newpatient, the mapping M will provide a corresponding binary output thatdescribes which tests are recommended for this patient.

This problem also can be viewed as a multi-class classification problemwhere for each case C_(i), its label is defined according to which testgave the correct diagnosis. For example, one possible approach is asfollows. For each test, all the training cases are labeled according tothe accuracy of that test for that case. Then, four classifiers aretrained (one for each test) using any binary classification algorithm(e.g., SVMs, Decision Trees, Bayesian networks, etc.). When a newpatient is considered, the patient data is tested in the fourclassifiers to predict which tests will give the correct diagnosis.

It is to be noted that with the above two approaches, the outcome of theprocess can be more than one test.

Another exemplary approach is as follows. Assume that there are m casesin a training set. A new case will be compared to these m cases usingthe n features described above. Based on this comparison, p cases areselected as being most “similar” to the current case, wherein similaritycan be defined in one of various ways. For instance, one approach is toconsider the Euclidean distance in the n-dimensional feature space.Other well-known distance measures can also be employed. It is to beappreciated that the above process can also be used to select exemplarcases from a library of cases for display as well.

One the similarity measures are determined and the most “similar” casesare identified, classifiers can be constructed for each of the k testsin the training set. In particular, by way of example, a classifierwould be constructed to test whether a lesion is benign or malignantusing, for example, each of the following sets of information: (i)current information and a diagnostic mammogram; (ii) current informationand ultrasound; (iii) current information and MRI, etc.

Each classifier would be constructed without learning from one of the pcases (i.e. leave-one-out approach), and then the withheld case would beclassified using this classifier. This would be repeated for each of thep cases, and the entire process for each of the k tests. An averagelikelihood would then be computed for each of the k tests, which wouldserve as the score of which test would be most useful.

It is to be appreciated that in accordance with other exemplaryembodiments of the invention, CAD systems may be implemented in adistributed model, wherein various modules/components of the CAD aredistributed over a communications network. For example, a CAD system canbe offered by an ASP (application service provider) to provide remoteaccess serving of CAD functions via an application server. For example,a database of cases used to identify similar cases could be located in acentral location. The advantage is that large databases of cases, whichoccupy a lot of memory, do not have to reside at every system. Inaddition, updates to the cases can be made very easily. This centrallocation could be within a hospital, for example, or it could be onecentral database accessed by everyone using such a system. Anotherpossibility is to use a distributed database, where cases are located inmultiple locations but are searched and accessed as if they are in oneplace. That way, cases located at different locations can be searched tofind similar cases. In addition to the database, the other parts of theCAD system, such as the classifier, could be centrally located.

Moreover, in view of above, it is to be appreciated that a CAD systemaccording to the invention can be implemented as a service (e.g., Webservice) that is offered by a third-party service provider pursuant toservice contract or SLA (service level agreement) to provide diagnosticsupport and other decision support functions as described herein basedone of various service/payment schemes. For example, the third-partyservice provider can be contractually obligated to train, maintain, andupdate classification models for various clinical domains, and aphysician or healthcare organization can access the CAD system “on-line”on a pay-per use basis, yearly subscription fee, etc. In such instance,various methods known to those of ordinary skill in the art can beimplemented to maintain patient confidentiality and otherwise transmitpatient data over communication channels using secured encryption,compression schemes, etc. Those of ordinary skill in the art can readilyenvision various architectures and implementation for CAD systemsaccording to the invention and nothing herein shall be construed as alimitation of the scope of the invention.

Although exemplary embodiments of the present invention have beendescribed herein with reference to the accompanying drawings, it is tobe understood that the invention is not limited to those preciseembodiments, and that various other changes and modifications may beaffected therein by one skilled in the art without departing from thescope or spirit of the invention. All such changes and modifications areintended to be included within the scope of the invention as defined bythe appended claims.

1. A method for providing automatic diagnosis and decision support forbreast imaging in an imaging process apparatus, comprising: obtaininginformation from image data of abnormal breast tissue of a subjectpatient; obtaining information from non-image data records of thesubject patient by extracting the non-image information from anunstructured free-text source using phrase spotting; and automaticallydiagnosing the abnormal breast tissue using the obtained informationfrom image data and the extracted non-image information.
 2. The methodof claim 1, wherein obtaining information from image data comprisesautomatically extracting one or more features from ultrasound imagedata, which are to be used for automatically diagnosing the abnormalbreast tissue.
 3. The method of claim 2, wherein the ultrasound data is3D ultrasound data.
 4. The method of claim 2, further comprisingautomatically detecting the abnormal breast tissue in the ultrasoundimage data prior to automatically extracting.
 5. The method of claim 2wherein automatically extracting one or more features from ultrasoundimage data comprises extracting features related to one of spiculationacoustic shadowing, or height/depth ratio of the abnormal breast tissue,and any combination of said features.
 6. The method of claim 2, whereinautomatically extracting one or more features from ultrasound image datacomprises extracting features related to one of acoustic, elastic,perfusion or 3D features of the abnormal breast tissue, or anycombination of said features.
 7. The method of claim 2, wherein theimage data comprises one of ultrasound image data, X-ray mammographyimage data, MRI image data, and a combination thereof.
 8. The method ofclaim 1, wherein automatically diagnosing the abnormal breast tissuecomprises automatically determining a probability of diagnosis.
 9. Themethod of claim 1, further comprising automatically identifying one ormore previously diagnosed cases that are similar to a current diagnosis.10. The method of claim 1, wherein automatically diagnosing the abnormalbreast tissue using the obtained information comprises classifying theabnormal breast tissue using a machine learning method, a model-basedmethod, or any combination of machine learning and model-based methods,that is trained to analyze the obtained information.
 11. The method ofclaim 8, further comprising automatically determining a confidence ofsaid probability of diagnosis.
 12. The method of claim 11, furthercomprising automatically determining additional information which wouldincrease the confidence of said probability of diagnosis.
 13. The methodof claim 12, wherein automatically determining additional informationfurther comprises determining a measure of usefulness of said additionalinformation in increasing said confidence of diagnosis.
 14. The methodof claim 9, wherein automatically identifying one or more previouslydiagnosed cases that are similar to a current diagnosis comprises usingthe obtained information to search a library of labeled cases withfeatures similar to the obtained information.
 15. The method of claim14, comprising displaying the identified cases.
 16. The method of claim10, comprising retraining the method for classifying on a continual orperiodic basis using expert data and/or data learned from a plurality ofcase studies.
 17. A method for providing automatic diagnosis anddecision support for breast imaging in an imaging process apparatus,comprising: obtaining information from ultrasound image data of abnormalbreast tissue of a subject patient; obtaining information from non-imagedata records of the subject patient by extracting the non-imageinformation from an unstructured free-text source using phrase spotting;automatically extracting features from the ultrasound image data, theextracted features comprising at least two of strain, fluid motion,blood motion, and B-mode images wherein the ultrasound image datacomprises 3D ultrasound image data and further comprising extractingfeatures from the 3D ultrasound image data; and automatically diagnosingthe abnormal breast tissue using the extracted features and theextracted non-image information.
 18. A method for providing automateddiagnostic and decision support for medical imaging in an imagingprocess apparatus, comprising: automatically extracting features frompatient data of a subject patient, the patient data comprising imagedata and non-image data, wherein extraction of non-image data isperformed from an unstructured free-text source using phrase spotting;and automatically determining a current state of the subject patient byanalyzing the features extracted from the patient data of the subjectpatient; and automatically providing decision support to assistphysician workflow regarding a healthcare or diagnostic or therapeuticpath for the subject patient, based on a determined current state of thesubject patient.
 19. The method of claim 18, wherein automaticallydetermining a current state of the subject patient comprisesautomatically determining a probability of diagnosis of breast cancer orthe probability of developing breast cancer in the future.
 20. Themethod of claim 19, wherein automatically providing decision supportcomprises automatically determining one or more additional imagingmethods that would increase a confidence of said probability ofdiagnosis of breast cancer.
 21. The method of claim 19, whereinautomatically providing decision support comprises automaticallyidentifying one or more previously diagnosed cases that are similar tothe current case.
 22. The method of claim 18, wherein automaticallydetermining a current state of the subject patient comprisesautomatically determining a likelihood of the subject patient developingbreast cancer.
 23. The method of claim 18, wherein automaticallyproviding decision support comprises automatically determining one ormore diagnostic imaging screening tests that could be given to thesubject patient for detecting breast cancer or provide a betterassessment of the likelihood of the subject patient developing breastcancer.
 24. The method of claim 18, wherein automatically determining acurrent state of the subject patient comprises automatically assessingcharacteristics and location of a breast tumor.
 25. The method of claim24, wherein automatically providing decision support comprisesautomatically determining one or more diagnostic imaging tests thatcould be used for staging the lesion.
 26. The method of claim 18,wherein automatically providing decision support to assist a physicianworkflow comprises suggesting a therapy.
 27. The method of claim 19,wherein automatically providing decision support comprises automaticallydetermining one or more additional features which would increase aconfidence of said probability of diagnosis of breast cancer.
 28. Themethod of claim 20, wherein automatically determining one or moreadditional imaging methods further comprises determining for each ofsaid one or more additional imaging methods, a measure of usefulness inincreasing said confidence of diagnosis.
 29. The method of claim 21,comprising displaying the identified cases.
 30. The method of claim 25,wherein automatically determining one or more diagnostic imaging teststhat could be used for staging the lesion further comprises providing ameasure of usefulness for each of the diagnostic imaging tests.
 31. Themethod of claim 25, wherein automatically providing decision supportcomprises automatically identifying one or more previous staging casesthat are similar to a current case.
 32. The method of claim 26, furthercomprising automatically providing a probability and/or measure ofconfidence that the therapy will have a determined outcome.
 33. Themethod of claim 26, further comprising automatically providing aprobability and/or measure of confidence that the therapy will not havea determined detrimental impact such as side effects.
 34. The method ofclaim 27, wherein automatically determining one or more additionalfeatures further comprises determining for each of said one or moreadditional features, a measure of usefulness in increasing saidconfidence of diagnosis.
 35. The meth of claim 31, further comprisingdisplaying the identified staging cases.
 36. The method of claim 26,wherein the probability is a distribution over possible outcomes bothbeneficial and detrimental.
 37. The method of claim 36, wherein theprobability is a set of distributions over possible outcomes bothbeneficial and detrimental at one or more time points in the future. 38.The method of claim 36, wherein the probability is a time-varyingdistribution over possible outcomes at different times in the future.